1. Field of the Invention
The video portion of the invention relates generally to an apparatus for measuring the video delay and perceptual degradation in video quality from a video source to a video destination. The video portion of the invention may be used to measure the video quality and video delay of transmission channels. The video may include moving images as well as still images. The transmission channels may include, but are not limited to, digital encoders and decoders, video storage/retrieval systems, analog transmission circuits, and digital transmission circuits. The new portion of the invention relates generally to an apparatus for non-intrusively measuring the audio delay and the perceptual change in audio-visual synchronization from a source of audio-visual information to a destination of audio-visual information via a transmission channel. Audio-visual synchronization is important because a loss of synchronization due to a transmission channel affects the perceived quality of the audio-visual information. Since the invention can non-intrusively measure the audio-visual synchronization without the use of special test signals, the audio-visual signal being transmitted by the transmission channel may include any arbitrary audio and video information. The transmission channels may include, but are not limited to, digital encoders and decoders, audio-video storage/retrieval systems, analog transmission circuits, and digital transmission circuits. The perception-based video quality measurement system of the parent application is used in implementing the present invention.
2. Description of the Prior Art
Devices for measuring the video quality of analog transmission channels have been available for a number of years. These devices utilize standard test patterns (such as a resolution test chart) or test waveforms (such as a color bar) for determining the degradation in video quality. Often, the results of these tests are not easily related to the perceptual may require the transmission channel to be taken out-of-service. Broadcasters have circumvented the out-of-service problem by using portions of the video signal that are not visible to the viewer (such as the vertical interval in the NTSC video standard) for quality testing.
With the advent of modern video coding systems, video signals are now commonly transmitted and stored in compressed digital form, with a potential loss of quality resulting from the compression process itself or from the decreased resilience of the compressed data when transmission errors occur. These digital systems have introduced many new impairments that are not quantified by the traditional methods mentioned above. Examples include unnatural (jerky) motion, image persistence, and image blocking. The basic reason the traditional quality measurement methods do not work for these new digital systems is that the perceptual quality of the destination video changes as a function of the pictorial content of the source video. Perceivable impairments may result when the source video is transmitted using a bandwidth that is less than the inherent spatial and temporal information content of the source video. Thus, one video scene (with low spatial and temporal information content) may be transmitted with small impairments while another video scene (with high spatial and temporal information content) may suffer large impairments. Attempts to measure the video quality using special test signals or test patterns with spatial and temporal information content that differ from the actual source video content results in inaccurate measurements. In, addition, since digital systems quite often either do not transmit the non-visible portions of the video or treat the non-visible portions differently than the visible portions, the transmission channel must be taken out of service to use special test patterns. Here, the visible portion of the video signal is that part of the picture that the viewer sees while the non-visible portion of the video signal is that part of the picture that the viewer does not see.
Designers of modern video coders and decoders have realized the shortfalls of the traditional analog methods mentioned above. Lacking anything better, they have quite often used the mean squared error (or some variant thereof) between the source video and the destination video for optimizing the video quality that their coders and decoders produce. However, the mean squared error does not correlate well with the perceptual quality of the video. In addition, since a perfect copy of the source and destination video is required to compute the mean squared error, this method is generally not practical unless the video source and video destination are geographically co-located.
Designers of video transmission equipment and standards organizations have resorted to using subjective tests when they require accurate measurements of video quality. Subjective tests are normally performed by having a large panel of viewers judge the perceived video quality. However, these subjective tests are very expensive and time consuming to conduct.
If the transmission channel is used for interactive (two-way) communications, then video delay is another important quality attribute of the transmission channel. Excessive delays can impede communications. In general, the video delay is also a function of the source video in modem digital communications systems. Consequently, the video delay of the transmission channel could change depending upon the information content of the source video. Thus, the use of special video delay test signals which are not the same as the actual video observed by the viewer could lead to erroneous results.
Devices for measuring the audio delay and the audio-visual synchronization of transmission channels have been available for a number of years. These devices utilize special audio or audio-visual test signals which require the transmission channel to be taken out-of-service (e.g. an audio-visual test signal such as a clapper or metronome where flashes of light in the video occur at the same time as clicks in the audio). These special audio and audio-visual synchronization test signals have proved quite useful for analog transmission channels.
The recent advent of digital transmission channels that utilize data compression techniques to remove information redundancy in the audio-visual signal has resulted in a new class of transmission channels where both the audio and video delays vary (and hence the audio-visual synchronization can vary) as the input signal is changed. For instance, the video delay of a scene that contains a small amount of motion (such as a head and shoulders scene with only the lips and eyes moving) may be quite different from the video delay of a scene that contains a large amount of motion (such as a football game). As a result, the audio-visual synchronization measured with traditional test signals is not easily related to the perceptual audio-visual synchronization of the video when that video is something other than the test signal itself. This can lead to inaccurate and irrelevant measurement results.
Another drawback with traditional audio and audio-visual synchronization test signals is that the transmission channel normally must be taken out-of-service, as the transmission channel is being used to transmit the test signal and thus cannot be used to transmit other audio-visual information.